Modeling peer assessment as a personalized predictor of teacher's grades: The case of OpenAnswer

Questions with open answers are rarely used as e-learning assessment tools because of the resulting high workload for the teacher/tutor that should grade them. This can be mitigated by having students grade each other's answers, but the uncertainty on the quality of the resulting grades could be high. In our OpenAnswer system we have modeled peer-assessment as a Bayesian network connecting a set of sub-networks (each representing a participating student) to the corresponding answers of her graded peers. The model has shown good ability to predict (without further info from the teacher) the exact teacher mark and a very good ability to predict it within 1 mark from the right one (ground truth). From the available datasets we noticed that different teachers sometimes disagree in their assessment of the same answer. For this reason in this paper we explore how the model can be tailored to the specific teacher to improve its prediction ability. To this aim, we parametrically define the CPTs (Conditional Probability Tables) describing the probabilistic dependence of a Bayesian variable from others in the modeled network, and we optimize the parameters generating the CPTs to obtain the smallest average difference between the predicted grades and the teacher's marks (ground truth). The optimization is carried out separately with respect to each teacher available in our datasets, or respect to the whole datasets. The paper discusses the results and shows that the prediction performance of our model, when optimized separately for each teacher, improves against the case in which our model is globally optimized respect to the whole dataset, which in turn improves against the predictions of the raw peer-assessment. The improved prediction would allow us to use OpenAnswer, without teacher intervention, as a class monitoring and diagnostic tool

Towards a quantitative evaluation of the relationship between the domain knowledge and the ability to assess peer work

In this work we present the preliminary results provided by the statistical modeling of the cognitive relationship between the knowledge about a topic a the ability to assess peer achievements on the same topic. Our starting point is Bloom's taxonomy of educational objectives in the cognitive domain, and our outcomes confirm the hypothesized ranking. A further consideration that can be derived is that meta-cognitive abilities (e.g., assessment) require deeper domain knowledge

Improved computation of individual ZPD in a distance learning system

This paper builds upon theoretical studies in the field of social constructivism. Lev Vygotsky is considered one of the greatest representatives of this research line, with his theory of the Zone of Proximal Development (ZPD). Our work aims at integrating this concept in the practice of a computer-assisted learning system. For each learner, the system stores a model summarizing the current Student Knowledge (SK). Each educational activity is specified through the deployed content, the skills required to tackle it, and those acquired, and is further annotated by the effort estimated for the task. The latter may change from one student to another, given the already achieved competence. A suitable weighting of the robustness (certainty) of student’s skills, stored in SK, and their combination are used to verify the inclusion of a learning activity in the student’s ZPD. With respect to our previous work, the algorithm for the calculation of the ZPD of the individual student has been optimized, by enhancing the certainty weighting policy, and a graphical display of the ZPD has been added. Thanks to the latter, the student can get a clear vision of the learning paths that he/she can presently tackle. This both facilitates the educational process, and helps developing the metacognitive ability self-assessment

Peer assessment and knowledge discovering in a community of learners

Thanks to the exponential growth of the Internet, Distance Education is becoming more and more strategic in many fields of daily life. Its main advantage is that students can learn through appropriate web platforms that allow them to take advantage of multimedia and interactive teaching materials, without constraints neither of time nor of space. Today, in fact, the Internet offers many platforms suitable for this purpose, such as Moodle, ATutor and others. Coursera is another example of a platform that offers different courses to thousands of enrolled students. This approach to learning is, however, posing new problems such as that of the assessment of the learning status of the learner in the case where there were thousands of students following a course, as is in Massive On-line Courses (MOOC). The Peer Assessment can therefore be a solution to this problem: evaluation takes place between peers, creating a dynamic in the community of learners that evolves autonomously. In this article, we present a first step towards this direction through a peer assessment mechanism led by the teacher who intervenes by evaluating a very small part of the students. Through a mechanism based on machine learning, and in particular on a modified form of K-NN, given the teacher’s grades, the system should converge towards an evaluation that is as similar as possible to the one that the teacher would have given. An experiment is presented with encouraging results

A human computer interactions framework for biometric user identification

Computer assisted functionalities and services have saturated our world becoming such an integral part of our daily activities that we hardly notice them. In this study we are focusing on enhancements in Human-Computer Interaction (HCI) that can be achieved by natural user recognition embedded in the employed interaction models. Natural identification among humans is mostly based on biometric characteristics representing what-we-are (face, body outlook, voice, etc.) and how-we-behave (gait, gestures, posture, etc.) Following this observation, we investigate different approaches and methods for adapting existing biometric identification methods and technologies to the needs of evolving natural human computer interfaces

Supporting mediated peer-evaluation to grade answers to open-ended questions

We show an approach to semi-automatic grading of answers given by students to open ended questions (open answers). We use both peer-evaluation and teacher evaluation. A learner is modeled by her Knowledge and her assessments quality (Judgment). The data generated by the peer- and teacher- evaluations, and by the learner models is represented by a Bayesian Network, in which the grades of the answers, and the elements of the learner models, are variables, with values in a probability distribution. The initial state of the network is determined by the peer-assessment data. Then, each teacher’s grading of an answer triggers evidence propagation in the network. The framework is implemented in a web-based system. We present also an experimental activity, set to verify the effectiveness of the approach, in terms of correctness of system grading, amount of required teacher's work, and correlation of system outputs with teacher’s grades and student’s final exam grade

Mobiles and wearables: owner biometrics and authentication

We discuss the design and development of HCI models for authentication based on gait and gesture that can be supported by mobile and wearable equipment. The paper proposes to use such biometric behavioral traits for partially transparent and continuous authentication by means of behavioral patterns. © 2016 Copyright held by the owner/author(s)

Adaptations and counteradaptations between the Screaming Cowbird (Molothrus rufoaxillaris) and the Baywing (Agelaioides badius)

Las aves parásitas de cría obligadas explotan el cuidado parental de individuos de otras especies (hospedadores) que crían a la progenie parásita a expensas de su propio éxito reproductivo. Los costos del parasitismo de cría seleccionan defensas antiparasitarias en la población hospedadora (i.e., adaptaciones que reducen el impacto del parasitismo). Esto, a su vez, puede favorecer contraadaptaciones en la población parásita, conduciendo a un proceso coevolutivo de tipo ?carrera armamentista? entre parásito y hospedador. En este trabajo se revisan las adaptaciones recíprocas entre un parásito especialista, el Tordo Pico Corto (Molothrus rufoaxillaris), y su principal hospedador, el Tordo Músico (Agelaioides badius). Las defensas del Tordo Músico incluyen el rechazo de las hembras parásitas, un comportamiento poco predecible de inicio de la puesta, el rechazo de la puesta completa en nidos superparasitados y la discriminación de juveniles que difieren en apariencia de los propios. Estas defensas son parcialmente contrarrestadas por el Tordo Pico Corto a través de un comportamiento elusivo y de una puesta rápida de huevos, un estrecho monitoreo de los nidos y la evolución de mimetismo visual y vocal del hospedador en las crías parásitas. Estos resultados sugieren una escalada armamentista entre el Tordo Pico Corto y el Tordo Músico que abarca todo el ciclo de nidificación. Futuros estudios de las interacciones entre estas especies antes, durante y después del parasitismo podrán mejorar la comprensión de los procesos coevolutivos parásito?hospedador y la evolución de la especialización en el uso de hospedadores en las aves parásitas de cría.Avian obligate brood parasites exploit the parental care of individuals of other species (hosts) that rear the parasitic offspring at the expense of their own reproductive success. The costs of parasitism select for antiparasitic defences in host populations (i.e., adaptations that reduce the impact of parasitism). This, in turn, may favour counteradaptations in the parasite population, leading to a coevolutionary arms race between parasite and host. We review the reciprocal adaptations between a specialist brood parasite, the Screaming Cowbird (Molothrus rufoaxillaris), and its primary host, the Baywing (Agelaioides badius). The defences of the Baywing include the rejection of parasitic females, a little predictable egg-laying behaviour, the rejection of whole "superparasitized" clutches, and discrimination against juveniles that do not resemble their own. These defences are partially countered by the Screaming Cowbird through an elusive behaviour and rapid egg-laying, a close monitoring of host nesting activities and the evolution of visual and vocal mimicry of host young in the parasite juveniles. These results suggest an escalated arms race between the Screaming Cowbird and the Baywing through the entire nesting cycle. Future studies of the interactions between these species before, during and after parasitism might improve the understanding of host-parasite coevolutionary processes and the evolution of specialization in host use in brood parasitic birds.Fil: de Marsico, Maria Cecilia. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Ecología, Genética y Evolución; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Ecología, Genética y Evolución de Buenos Aires; ArgentinaFil: Reboreda, Juan Carlos. Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Ecología, Genética y Evolución; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Oficina de Coordinación Administrativa Ciudad Universitaria. Instituto de Ecología, Genética y Evolución de Buenos Aires; Argentin

CABALA: Collaborative Architectures based on Biometric Adaptable Layers and Activities

The lack of communication and of dynamic adaptation to working settings often hinder stable performances of subsystems of present multibiometric architectures. The calibration phase often uses a specific training set, so that (sub)systems are tuned with respect to well determined conditions. In this work we investigate the modular construction of systems according to CABALA (Collaborative Architectures based on Biometric Adaptable Layers and Activities) approach. Different levels of flexibility and collaboration are supported. The computation of system reliability (SRR), for each single response of each single subsystem, allows to address temporary decrease of accuracy due to adverse conditions (light, dirty sensors, etc.), by possibly refusing a poorly reliable response or by asking for a new recognition operation. Subsystems can collaborate at a twofold level, both in returning a jointly determined answer, and in co-evolving to tune to changing conditions. At the first level, single-biometric subsystems implement the N-Cross Testing Protocol: they work in parallel, but exchange information to reach the final response. At an higher level of interdependency, parameters of each subsystem can be dynamically optimized according to the behavior of their companions. To this aim, an additional Supervisor Module analyzes the single results and, in our present implementation, modifies the degree of reliability required from each subsystem to accept its future responses. The paper explores different combinations of these novel strategies. We demonstrate that as component collaboration increases, the same happens to both the overall system accuracy and to the ability to identify unstable subsystems. (C) 2011 Elsevier Ltd. All rights reserved